Skip to content

Conversation

@steven-pribilinskiy
Copy link

@steven-pribilinskiy steven-pribilinskiy commented Oct 24, 2025

Overview

This PR exposes the complete Claude CLI metadata that was previously being filtered out by the SDK's parseMessage() method.

After analyzing the actual CLI output (claude --verbose --output-format stream-json), we discovered 35 missing features across three categories:

  1. Fields defined in types but never populated (type safety bug)
  2. Fields provided by CLI but not in types (type completeness bug)
  3. System init data completely discarded (data loss bug)

What's Included

Extended Type Definitions

ResultMessage & CLIResultOutput:

  • Added ModelUsageInfo interface for per-model breakdown
  • Extended with performance metrics: duration_ms, duration_api_ms, num_turns
  • Added error indicators: is_error (boolean), result (simplified string access)
  • Extended usage with service_tier, server_tool_use.web_search_requests
  • Added cache tier breakdown: ephemeral_5m_input_tokens, ephemeral_1h_input_tokens
  • Fixed cost mapping: now includes full breakdown (input/output/cache costs, not just total)
  • Added model usage: modelUsage (per-model token counts, costs, web searches, context window)
  • Added permission tracking: permission_denials array
  • Added request tracking: uuid field

SystemMessage:

  • Extended to expose full system init capabilities
  • Includes: tools, MCP servers, agents, skills, slash commands, model info, CLI version

Field Additions (32+ total)

Performance Metrics (3):

  • duration_ms - Total query duration
  • duration_api_ms - API response time
  • num_turns - Number of conversation turns

Error & Result (2):

  • is_error - Boolean error status
  • result - Simplified result string

Token Usage (8):

  • input_tokens, output_tokens (existing)
  • cache_creation_input_tokens, cache_read_input_tokens (existing)
  • service_tier - Service tier information
  • server_tool_use.web_search_requests - Web search count
  • cache_creation.ephemeral_5m_input_tokens - 5-minute cache tier
  • cache_creation.ephemeral_1h_input_tokens - 1-hour cache tier

Cost Breakdown (5):

  • input_cost, output_cost
  • cache_creation_cost, cache_read_cost
  • total_cost

Model Usage (per model, 6 fields):

  • inputTokens, outputTokens
  • cacheReadInputTokens, cacheCreationInputTokens
  • webSearchRequests - Web searches per model
  • contextWindow - Model's context window size
  • costUSD - Cost per model

Tracking (2):

  • permission_denials - Array of denied permissions
  • uuid - Request UUID for correlation

System Capabilities (12):

  • cwd, model, claude_code_version
  • permissionMode, apiKeySource, output_style
  • tools (19 available), mcp_servers (with connection status)
  • slash_commands (23 available), agents (19 available)
  • skills, uuid

New ResponseParser Methods (7 total)

Performance & Tracking:

  • getPerformanceMetrics() - Access timing and turn information
  • getRequestUUID() - Get request UUID for correlation

Permissions & Audit:

  • getPermissionDenials() - Check which permissions were denied

Capabilities Discovery:

  • getSystemCapabilities() - Get available tools, agents, MCP servers, etc.

Cost & Usage Analytics:

  • getModelUsageBreakdown() - Access per-model usage statistics
  • getWebSearchUsage() - Track web searches (total + per-model breakdown)
  • getCacheBreakdown() - Analyze cache tier usage (5m vs 1h)

Type Exports

All new types properly exported from src/index.ts:

  • ModelUsageInfo
  • PerformanceMetrics
  • SystemCapabilities
  • WebSearchUsage
  • CacheBreakdown
  • ToolExecution
  • UsageStats

Backwards Compatibility

Fully backwards compatible - All new fields are optional
✅ All existing methods continue to work unchanged
✅ New methods are purely additive
✅ No breaking changes to any interfaces

Implementation Details

parseMessage() Changes (src/_internal/client.ts)

  • System messages now exposed with full capabilities (previously returned null)
  • Result messages include ALL fields from CLI output (not just subset)
  • Added extraction of: is_error, result, enhanced usage, full cost, modelUsage
  • Assistant and error messages unchanged

Type Safety Fixes

  • Fixed type-implementation mismatches (types now match runtime)
  • Added missing fields that CLI provides: is_error, result, web search tracking, cache tiers
  • Extended ModelUsageInfo with webSearchRequests and contextWindow
  • Fixed cost mapping to preserve full breakdown (was only extracting total_cost_usd)

Use Cases

// Get performance insights
const perf = await parser.getPerformanceMetrics();
console.log(`Query took ${perf.durationMs}ms in ${perf.numTurns} turns`);

// Check permission usage
const denied = await parser.getPermissionDenials();
if (denied.length > 0) {
  console.log(`Denied tools: ${denied.join(', ')}`);
}

// Get system capabilities
const caps = await parser.getSystemCapabilities();
console.log(`Running CLI v${caps.claudeCodeVersion}`);
console.log(`Model: ${caps.model}`);
console.log(`Available tools (${caps.tools.length}): ${caps.tools.join(', ')}`);
console.log(`MCP servers: ${caps.mcpServers.map(s => `${s.name} (${s.status})`).join(', ')}`);

// Per-model cost tracking with web search counts
const usage = await parser.getModelUsageBreakdown();
for (const [model, stats] of Object.entries(usage || {})) {
  console.log(`${model}:`);
  console.log(`  Tokens: ${stats.inputTokens + stats.outputTokens}`);
  console.log(`  Web searches: ${stats.webSearchRequests}`);
  console.log(`  Context window: ${stats.contextWindow}`);
  console.log(`  Cost: $${stats.costUSD}`);
}

// Web search usage tracking
const webSearch = await parser.getWebSearchUsage();
console.log(`Total web searches: ${webSearch.total}`);
console.log(`By model:`, webSearch.byModel);

// Cache tier breakdown
const cache = await parser.getCacheBreakdown();
console.log(`Cache: ${cache.ephemeral5m} tokens (5m), ${cache.ephemeral1h} tokens (1h)`);

Testing

✅ Code compiles without TypeScript errors
✅ All new types properly defined and exported
✅ ResponseParser methods properly implemented
✅ No linter errors
✅ Backwards compatible with existing code
✅ All fields extracted from CLI output

Commits

  1. 7c0f759: Initial implementation (26 fields + 5 methods)

    • Performance metrics, model usage, permission tracking, system capabilities
    • 5 convenience methods for common use cases
  2. b040584: Gap filling (6 additional fields + 2 methods + 4 exports)

    • Error indicators, web search tracking, cache tier breakdown
    • Full cost mapping fix
    • 2 additional convenience methods
    • Proper type exports

Resolves

Closes #12

Summary Table

Category Before After Fields Added
Type Safety ❌ Types lie about available fields ✅ Types match runtime Type safety restored
Performance Metrics ❌ Defined but never populated ✅ Fully available 3 fields
Error Indicators ❌ Not available ✅ Available 2 fields
Token Usage (basic) ⚠️ Partial ✅ Complete 4 fields (enhanced)
Token Usage (enhanced) ❌ Not available ✅ Available 4 new fields
Cost Breakdown ⚠️ Only total cost ✅ Full breakdown 5 fields
Model Usage ❌ Defined but never populated ✅ Fully available 6 fields per model
Permission Tracking ❌ Defined but never populated ✅ Fully available 1 field
Request Tracking ❌ Defined but never populated ✅ Fully available 1 field
System Capabilities ❌ Completely discarded ✅ Fully exposed 12 fields
Web Search Tracking ❌ Not available ✅ Available 2 fields
Cache Analytics ❌ Not available ✅ Available 2 fields
Convenience Methods 2 basic 7 comprehensive +5 methods
Backwards Compatibility N/A ✅ 100% maintained No breaking changes

Total: 32+ fields added, 7 convenience methods, complete type safety

- Extended ResultMessage with performance metrics (duration_ms, duration_api_ms, num_turns)
- Added modelUsage field for per-model token and cost breakdown
- Exposed permission_denials and uuid for auditing and request tracking
- Updated SystemMessage to expose full system init capabilities
- Modified parseMessage() to include all metadata fields instead of filtering

New ResponseParser convenience methods:
- getPerformanceMetrics(): access duration and turn count
- getPermissionDenials(): check denied permissions
- getSystemCapabilities(): get tools, agents, MCP servers, etc.
- getModelUsageBreakdown(): access per-model usage stats
- getRequestUUID(): get request UUID for tracking

Backwards compatible - all new fields are optional

Resolves instantlyeasy#12
Complete the metadata exposure by adding all fields from CLI output:

Type Extensions (ResultMessage & CLIResultOutput):
- Add is_error field for error status checking
- Add result field as simplified content access
- Add service_tier to usage stats
- Add server_tool_use.web_search_requests for web search tracking
- Add cache_creation breakdown (ephemeral_5m/1h_input_tokens)
- Extend cost object with full breakdown (input/output/cache costs)

Implementation Fixes:
- Update parseMessage() to extract is_error and result fields
- Fix cost mapping to preserve full cost breakdown (not just total_cost_usd)
- Pass through all usage fields including server_tool_use and cache_creation

New ResponseParser Methods:
- getWebSearchUsage(): track web searches (total and per-model)
- getCacheBreakdown(): analyze cache tier usage (5m vs 1h)

Public API Exports:
- Export PerformanceMetrics type
- Export SystemCapabilities type
- Export WebSearchUsage type
- Export CacheBreakdown type

This completes the type safety fix by ensuring all CLI-provided fields
are properly defined, extracted, and exposed to SDK users. Addresses
the remaining gaps discovered through CLI output analysis.

Related to instantlyeasy#12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Expose full CLI metadata (duration, modelUsage, permissions)

1 participant